Serialising Complex Objects

Pass the class

Dataset uses JSON for serialisation by default. This works fine for simple calcuations, however sometimes it is inconvenient (or impossible) to rely purely on JSON serialisable objects.

This can be easily demonstrated by creating a function that creates a class:

def classcreate(name):
    class tmp:

        def __init__(self, name):
            self.name = name

    return tmp(name)

test = classcreate("test")

print(test.name)

test

This works just fine for us locally, however if we tried to wrap this into a Dataset, we’d get a JSON error.

Swapping out the serialiser

There are currently 4 inbuilt serialisers within remotemanager.serialisation you can swap to:

• serialjson (default)

• serialyaml

• serialdill*

• serialjsonpickle*

Those marked with a * are able to serialise complex objects.

To swap out the serialiser, simply import them and pass them at dataset creation, as you would a URL:

from remotemanager import Dataset
from remotemanager.serialisation import serialdill

ser = serialdill()

ds = Dataset(function=classcreate,
                 serialiser=ser,
                 skip=False)

ds.append_run({"name": "dilltest"})

ds.run()

ds.wait(1, 10)

ds.fetch_results()

ds.results[0].name

appended run runner-0
Staging Dataset... Staged 1/1 Runners
Transferring for 1/1 Runners
Transferring 5 Files... Done
Remotely executing 1/1 Runners
Fetching results
Transferring 2 Files... Done

'dilltest'

Requirements

For these serialisers to work, you must have their prerequisite package installed:

serialdill requires pip install dill serialjsonpickle requires pip install jsonpickle

Creating your own serialiser

If none of these options suit your use case, there remains the possibility of creating your own class. To do so, you should subclass ../remotemanager.serialisation.serial docs link here, implementing the methods as seen in the various subclasses of serial yaml, for example